Methods and apparatus to improve feature engineering efficiency with metadata unit operations

ABSTRACT

Methods, apparatus, systems and articles of manufacture are disclosed to improve feature engineering efficiency. An example method disclosed herein includes retrieving a log file in a first file format, the log file containing feature occurrence data, generating a first unit operation based on the first file format to extract the feature occurrence data from the log file to a string, the first unit operation associated with a first metadata tag, generating second unit operations to identify respective features from the feature occurrence data, the second unit operations associated with respective second metadata tags, and generating a first sequence of the first metadata tag and the second metadata tags to create a first vector output file of the feature occurrence data

RELATED APPLICATION

This patent arises from a continuation of U.S. patent application Ser.No. 15/280,044, filed on Sep. 29, 2016, which is hereby incorporated byreference in its entirety.

FIELD OF THE DISCLOSURE

This disclosure relates generally to malware detection, and, moreparticularly, methods and apparatus to improve feature engineeringefficiency with metadata unit operations.

BACKGROUND

In recent years, malware analysis entities have enjoyed access tobehavior data from computing devices so that behavior log files may begenerated. Malware analysis entities include businesses that studybehaviors of programs that execute on devices, such as network accessattempts and/or other device function invocations. In some examples, themalware analysis entities apply behavior log files to one or moremachine learning systems so that predictive patterns may be identifiedin an effort to prevent malware operations before such malware can causedamage and/or propagate further within one or more computing devices.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic illustration of an example computing environmentconstructed in accordance with the teachings of this disclosure toimprove feature engineering efficiency with metadata unit operations.

FIGS. 2A and 2B are example text log input file formats generated bybehavior aggregators that are processed by the example computingenvironment of FIG. 1.

FIG. 2C is an example vector output file generated by the examplecomputing environment of FIG. 1.

FIG. 3 is a schematic illustration of an example feature engineeringsystem of FIG. 1.

FIGS. 4-6 are flowcharts representative of example machine readableinstructions that may be executed to implement the example featureengineering system of FIGS. 1 and/or 3.

FIG. 7 is a block diagram of an example processor platform structured toexecute the example machine readable instructions of FIGS. 4-6 toimplement the example feature engineering system of FIGS. 1 and/or 3.

DETAILED DESCRIPTION

Malware analysis entities include organizations that design softwareand/or hardware applications to protect computing devices and/ornetworks from malicious threats. In some examples, the malware analysisentities distribute virus and/or malware detection and containmentapplications to be installed on client devices that detect suspiciousdata, files and/or behaviors before causing damage to the computingdevice. Additionally, the malware analysis entities may employ“sandboxing” techniques by running malware or suspected malware on adedicated set of hardware or in a virtual machine/emulator. As such, any“dirty” actions are confined in a safe manner. The malware analysisentities may also collect behavior information of the devices undertheir monitoring purview, regardless of whether that behaviorinformation includes malicious or non-malicious behavior data. Inparticular, the malware analysis entities utilize both maliciousbehavior data and non-malicious behavior data to facilitate featureengineering (e.g., with the aid of machine learning). Generallyspeaking, feature engineering collects feature data (e.g., features mayinclude, but are not limited to opening network connections, openinginput/output (I/O) bridges, registry access attempts, system propertyaccess attempts, program execution attempts, invoking libraries, etc.)that is used as input to modeling activities for machine learningalgorithms, in which output from the machine learning process may revealtrends, signals, and/or sequences that aid in early malware detection.Feature data may include any type of information, such as computingbehavior data, which may aid in the ability to make predictions.

FIG. 1 is a schematic illustration of an example computing environment100 to improve feature engineering efficiency. In the illustratedexample of FIG. 1, the environment 100 includes one or more behavioraggregators 102, such as malware analysis entities (e.g., securitysandboxes, such as McAfee® Advanced Threat Defense, FireEye® ForensicAnalysis Platform, Mandiant®, etc.). The example behavior aggregators102 are communicatively connected to one or more network(s) 104, such aslocal area networks (LANs), wide area networks (WANs), the Internet,etc. As described above, hardware and/or software applications developedby the behavior aggregators 102 in the form of virus and/or malwareprotection may be installed and/or otherwise associated with one or morecomputing devices 106. The example computing devices 106 arecommunicatively connected to the example network 104 and may includepersonal computers (PCs), servers, mainframes, tablet devices, wirelesstelephones and/or other networked computing devices.

Behavior data associated with the example computing devices 106 of FIG.1 is retrieved and/or otherwise received by the example behavioraggregators 102. Based on the identified behaviors, the example behavioraggregators generate log files that identify features that occur whenrespective programs execute on the example computing devices 106. FIG.2A illustrates an example input from an Android® program that wasgenerated by an example behavior aggregator 102 as a text log 200. Inthe illustrated example of FIG. 2A, the text log 200 includes a list ofchronologically/sequentially occurring features (events) associated withthe program (e.g., an Android® program) that executed on the device. Forexample, the feature “android.os.SystemProperties.get” occurred six (6)times (202), followed by the feature“android.telephony.TelephonyManager.getDeviceId” (204). Generallyspeaking, feature occurrences may include any number of differentpermutations that appear in different degrees of frequency and/or order.Some occurrences may be innocuous, while others may be indicative of athreat (e.g., virus, malware, etc.).

FIG. 2B illustrates an alternate example input from a Windows® programthat was generated by an example behavior aggregator 102 as a text log250. In the illustrated example of FIG. 2B, the text log 250 includessome differences from the example text log 200 of FIG. 2A. Inparticular, the example text log 250 of FIG. 2B includes a list ofchronologically/sequentially occurring features associated with aWindows® that executed on the device, in which the features includenomenclature that is different from the illustrated example of FIG. 2A.Additionally, the example features of the text log 250 of FIG. 2Binclude example procedure names surrounded by quotation marks 252,in-line names of executables surrounded by quotation marks 254 andlibrary nomenclature 256 (e.g., “SbieDll.dll”). While the illustratedexamples of FIGS. 2A and 2B include logs as text files, examplesdisclosed herein are not limited thereto. Example log files may be inany other format such as, for example JavaScript Object Notation (JSON),binary, Extensible Markup Language (XML), etc.

Traditional feature engineering techniques develop and apply separatevector creation programs that are unique to and/or otherwise customizedfor each type of log file that might be generated by respective behavioraggregators 102. The vector creation programs extract feature data froma respective log to create vectors to be used as a formatted input to amachine learning system 108, as shown in FIG. 1. As used herein,“vectors” are a collection of observed features associated with aprogram that has executed on a computing device, in which the program isidentified with a unique hash value. FIG. 2C illustrates an examplevector output file 270. In the illustrated example of FIG. 2C, theoutput file 270 includes rows, in which each program that has beenobserved to execute on a computing device is associated with a hash. Afirst hash value 272 of the example vector output file is shown in afirst column 274 for a first row 276. A second column 278 illustrates anexample dirty bit, in which a value of zero (“0”) is indicative of cleanexecution by an example program (e.g., no observed malware) and a valueof one (“1”) is indicative of dirty execution by an example program.

Following the example second column 278 containing the example dirtybit, the example vector output file 270 includes any number ofadditional columns to identify which features occurred during executionof the example program associated with the hash 272. A first identifiedfeature of the example first row 276 is “2:1,” and a second identifiedfeature is “27:1.” Each numeric value prior to the colon (:) representsa unique feature. In particular, a dictionary lists features andassociated vector values, thereby making the example vector output file270 smaller and properly formatted as an input file to the examplemachine learning system 108. For example, the feature “getDevicelD” maybe associated with vector value “2,” and the feature “getlnputStream”may be associated with vector value “27.” In some examples, a valueafter the colon (:) represents a value associated with the feature. Inthe event a feature value is not needed or is of no consequence, thevalue may be set as one (1). The example output file 270 is shown in theillustrated example of FIG. 2C as a particular format associated withLibrary for Support Vector Machines (LIBSVM), but examples disclosedherein are not limited thereto.

In the event the example text log 250 of FIG. 2B is newly retrievedand/or otherwise received for the first time, an associated vectorcreation program is developed to accommodate for particular formattingand nomenclature of the log 250. Similarly, in the event the exampletext log 200 of FIG. 2A is newly retrieved and/or otherwise received forthe first time, another associated vector creation program must bedeveloped to accommodate for the particular specifications, formattingand/or other nuances of the newly retrieved log. As such, malwareevaluation personnel chartered with the responsibility of researchingmalware behaviors must develop new vector creation programs as newand/or alternate behavior aggregators 102 emerge in the industry.Furthermore, in the event new features are developed by one or moreprograms, then the malware evaluation personnel must update any existingvector creation programs to accommodate for those new features, whichmay also include updating one or more dictionaries and/or regularexpression string repositories. In some cases, a new feature type willemerge for multiple platforms, thereby requiring the malware evaluationpersonnel to apply and/or otherwise engineer updates to multipledifferent vector creation programs. Failure to properly apply updatesmay lead to bias and error when inconsistent data input is provided tothe example machine learning system 108. Additionally, efforts bymalware evaluation personnel to develop, update and maintain the one ormore vector creation programs associated with different ones of the logfiles generated by different ones of the behavior aggregators 102 leadsto inefficiency and/or lost productivity.

Examples disclosed herein improve feature engineering efficiency.Generally speaking, an example feature engineering system 110 retrievesone or more feature log files that may have been generated by one ormore behavior aggregators 102 and converts and/or otherwise formats theminto vectors (e.g., rows of feature vectors in a particular formatcompatible with the example machine learning system. Unlike traditionaltechniques for creating vector output files, which include disparatevector creation programs developed for each input data type, eachprogram platform type and/or each program, the example featureengineering system 110 includes a single program to accommodate for anytype of feature, feature nomenclature and/or file data type. As such,any updates and/or management in connection with new and/or alternatefeatures or new and/or alternate feature nomenclature, the examplefeature engineering system 110 may be modified in a centralized mannerto create updated unit operations, updated operation flow sequence(s),updated dictionary management and/or updated regular expression stringmanagement.

In the illustrated example of FIG. 1, the machine learning system(s) 108and/or the feature engineering system 110 may be implemented with a bigdata framework platform 112. Generally speaking, amounts of aggregateddata generated by the example computing devices 106 and evaluated by theexample behavior aggregators 102 is large, in which such amounts of dataare typically too large for file systems associated with standardoperating systems (OSs). To accommodate data volumes associated with theexample environment 100 of FIG. 1, the big data framework platform 112may include a distributed file system, such as Hadoop. In particular,Hadoop® is a distributed file system (sometimes referred to as theHadoop Distributed File System (HDFS)) that enables storage anddistribution of data throughout disparate networked storage locations,in which each storage location may be on the order of petabytes in size.In the event one or more additional storage locations is to be added orremoved, the Hadoop® file system accommodates scalability functionalityto handle a growing, shrinking and/or otherwise dynamic set of filestorage locations. In some examples, the HDFS is installed on aWindows®-based server/platform or a Linux®-based server/platform.

Additionally, the example big data framework platform 112 may includeone or more advanced analytics systems, such as Spark®. Unlike Hadoop®,which applies a MapReduce system to transfer large amounts of datato/from physical storage locations, Spark® utilizes relatively fastermemory to perform one or more analytics on the available data. WhileSpark® still utilizes the underlying HDFS for storage at distributedlocations, Spark® facilitates machine learning algorithms on large datasets, such as machine learning algorithms to identify patterns in thedata. Additionally, Spark® includes computational features to scaleadditional platform resources when additional processing power isneeded. Similar to Hadoop®, Spark® may be installed on a Windows®-basedserver/platform or a Linux®-based server/platform. While the aboveexamples include Hadoop® and Spark® as candidate big data frameworkplatforms 112, examples disclosed herein are not limited thereto. Forexample, other file systems may be included with examples disclosedherein, such as MongoDB, Amazon's S3 system, etc.

FIG. 3 illustrates the example feature engineering system 110 of FIG. 1.In the illustrated example of FIG. 3, the feature engineering system 110includes a dictionary editor 302 communicatively connected to adictionary storage 304 and a regular expression storage 306. The exampledictionary editor 302 may populate the example dictionary storage 304and/or the example regular expression storage 306 to maintain paritywith dictionary information and regular expression information that maybe consistent with those utilized by the example machine learning system108. In some examples, the dictionary storage includes information thatassociates particular feature nomenclature with an integer value, asdescribed above. In still other examples, the dictionary storageincludes alternate nomenclature for the same type of feature to allowfeature nomenclature normalization when input logs by different behavioraggregators 102 refer to the same type of feature with differentnomenclature. For example, a first behavior aggregator may refer to afeature associated with opening a network connection by using thenomenclature “OpenNetworkConn,” while a second behavior aggregator mayrefer to a feature of the same functionality by using the nomenclature“Open TCPConn,” while a third behavior aggregator may refer to a featureof the same functionality by using the nomenclature “OpenUDPConn.” Whileall three of these disparate nomenclatures refers to the same programfunctionality, the example dictionary editor 302 updates the exampledictionary storage 304 to create associations therebetween.

Additionally, the example dictionary storage 304 and the example regularexpression storage 306 include one or more patterns of features and/orfeature values to be identified during feature engineering of an inputlog of interest. In other words, search terms. For example, atranslation unit operation (described in further detail below) may beused with a JSON log type and reference a target dictionary pattern“key1 key3.” In particular, the JSON input may include any number andtype of key, such as {“key1”:“val1”,“key2”:“val2”,“key3”:“val3”}. Basedon the target dictionary pattern “key1 key3,” the resulting output basedon the JSON log input is “val1, val3”.

Additionally or alternatively, the example regular expression storage306 may be invoked by the example dictionary editor 302 to apply searchterms to the input log of interest. While the illustrated exampledictionary editor 302 of FIG. 3 uses the term “dictionary,” the exampledictionary editor 302 may also access, use and/or modify operations ofthe example feature engineering system 110 using regular expressionsexclusively or in combination with one or more dictionaries. Generallyspeaking, regular expressions include strings and/or sequences ofdefined characters to create one or more search patterns. The regularexpression strings may include metacharacters indicative of Booleanoperations, grouping operations, quantification operations, etc. Forexample, in the event electronic mail (e-mail) addresses are believed tobe embedded in malware attempts, the example regular expression storage306 may be modified and/or otherwise configured to include the regularexpression “\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b”. Utilization ofthe aforementioned regular expression allows the extraction of e-mailaddresses from the candidate input log in a manner that is more agnosticto the input log type. For instance, the aforementioned regularexpression may be used for JSON log file types, XML log file types andtext log file types.

Returning to the illustrated example of FIG. 3, the feature engineeringsystem 110 also includes a unit operation builder 308, which includes afile to string operation builder 310, an extraction operation builder312, a vector space operation builder 314, a hashing operation builder316, a formatting operation builder 318 and a feature save operationbuilder 320. The example feature engineering system 110 also includes anexample operation flow builder 322 and an example log file retriever324. In operation, the example feature engineering system determineswhether to conduct configuration operations or whether a runtimescenario is occurring based on, for example, receipt of an input logfrom one or more behavior aggregators 102. For the sake of example, andassuming configuration operations are to occur based on, for example, auser prompt in view of a new input log type, the example dictionaryeditor updates the example dictionary storage 304 and/or updates theexample regular expression string(s) stored in the example regularexpression storage 306.

Updating the dictionary may include adding nomenclature associated witha log file received from a behavior aggregator 102, such as the exampletext log 200 of FIG. 2A and/or the example text log 250 of FIG. 2B. Forthe sake of example, if the retrieved and/or otherwise received log file200 of FIG. 2A has never been received before, then some or all of thefeatures therein may not yet be included in the example dictionarystorage 304. The example dictionary editor 302 parses the retrieved logfile for nomenclature to be added to the example dictionary storage 304and, in some examples, one or more feature terms/nomenclature may beadded manually (e.g., added by malware evaluation personnel). In someexamples, the dictionary editor 302 populates the dictionary storage 304with nomenclature terms and/or sequences of nomenclature terms. Forexample, to detect an occurrence of the feature“android.os.SystemProperties.get,” the example dictionary editor 302 mayadd a nomenclature string sequence to the example dictionary storage 304to look for each of the desired terms of the target string of interest,such as “android” followed by “os” followed by “SystemProperties”followed by “get.” Similarly, in the event regular expressions are to beused, the example dictionary editor 302 may develop expression searchstrings to identify the desired feature nomenclature.

In other examples, the dictionary editor 302 may generate nomenclaturestring terms to identify particular features of interest and/orparticular calls to executables of interest. Returning to theillustrated example of FIG. 2B, the dictionary editor 302 may include asearch string “popupkiller.exe” to identify instances of one or morefeatures that invokes a particular executable of interest. As describedabove, while examples disclosed herein refer to example dictionaries(e.g., dictionary storage 304), such examples are not limited theretoand may be considered in connection with regular expression stringsstored in the example regular expression storage 306.

While traditional approaches to handling the example text log 200 ofFIG. 2A, the example text log 250 of FIG. 2B and/or one or more futurelogs (e.g., JSON logs, binary logs, etc.) required development ofindividualized vector creation programs to parse and extract log datainto a vector output, examples disclosed herein facilitate ametadata-driven approach to processing retrieved and/or otherwisereceived log files without cumbersome and error prone development ofindividualized vector creation programs. As such, in the event a newfeature is identified, or a new log data type is identified (e.g., a newplain text log file, a new comma separated value log file, a new JSONlog file, etc.), examples disclosed herein facilitate a metadata-drivenapproach to handling new and/or modified input data types.

To facilitate the metadata-driven approach of feature engineering, theexample unit operation builder 308 builds one or more unit operations inconnection with a retrieved and/or otherwise received log (e.g., a loghaving a particular format (e.g., text, JSON, etc.) generated by anexample behavior aggregator 102. As used herein, a unit operation is ametadata identifier (e.g., a tag) associated with one or more processingtasks of a log of interest. Unit operations may includefunctions/procedures such as, but not limited to, pattern matching,string replacement, string extraction (e.g., from a log file), stringhashing, string translation, n-gram operation(s), pattern formatting,storage operation(s), etc. Table 1 below is an example list of candidateunit operations generated and/or modified by the example unit operationbuilder 308 of FIG. 3.

TABLE 1 Op ID Input/Output Description p01 Filename/string Convert atext file into one line of string text. p02 Filename, JSON Convert JSONfile into one paths/string line of string text. m01 String,pattern/string Find matching unit by regular expression. e01 String,pattern/string Extract substring from string by regular expression. h01String, hash Hash a string or substring to a function/number number. t01String, mapping Translate to different string dictionary/string from adictionary. n01 String array, n-gram N-Gram extraction. number/stringarray f01 Token array/string Format to a particular type (e g., LIBSVM).s01 Filename, string Save features to vector file.In the illustrated example of Table 1, the example operation builder 308generates a particular operation identifier (Op_ID) as a metadata tagthat is associated with underlying functions/procedures to process aportion of a retrieved and/or otherwise received log file. For example,in the event a text-type log file is received by the example featureengineering system 110, then the example file to string operationbuilder 310 may generate an example unit operation named “p01” toconvert that received text file into a string, thereby allowing furtherprocessing, as described below. Additionally, in the event a second logfile is received by the example feature engineering system 110 that isof type JSON, then the example file to string operation builder 310 maygenerate an example unit operation named “p02” to convert that receivedJSON file into a string. In other words, regardless of the type of logfile retrieved and/or otherwise received by the example featureengineering system 110, one or more unit operations may be generated bythe example file to string operation builder 310 to accommodate for theinput type (e.g., by applying known file structure standards of the filetype, such as XML tag formatting, JSON formatting, text data, etc.).

Additionally, the example extraction operation builder 312 generatesunit operation(s) to facilitate extraction, matching and/or translationof data from the retrieved log file. As described above, the exampledictionary storage 304 and/or the example regular expression storage 306may include one or more desired patterns that are to be searched when alog file is retrieved. In the illustrated example Table 1, a unitoperation (Op_ID) named “m01” takes a string as an input (e.g., thestring generated by the example “p01” unit operation) and a desiredpattern of interest as an input defined by the example dictionarystorage 304 or defined by the example regular expression storage 306,and generates an indication of a match when such a match of the desiredpattern is detected. Additionally, the example extraction operationbuilder 312 generates a unit operation (Op_ID) named “e01” to extract asubstring of the detected match.

In some examples, the extraction operation builder 312 generates a unitoperation to find one or more strings based on a key list from theexample dictionary storage 304 named “t01” (or named as metadata tag“t02,” “t03,” etc. depending on how many different log file types ofinterest are being handled). For example, the t01 unit operation tag maybe associated with functionality that is particularly helpful for logsof type JSON to get values for features that may have similarnomenclature, but refer to the same type of feature. As described above,consider a feature that is related to opening a network connection. Insome logs, this feature functionality is associated with thenomenclature “OpenTCPConn,” while in other logs this functionality isassociated with the nomenclature “OpenUDPConn,” while in still otherlogs this functionality is associated with the nomenclature“OpenNetworkConn.” As such, the example t01 unit operation generated bythe example extraction operation builder 312 normalizes log featurenomenclature.

The example vector space operation builder 314 generates one or moreunit operations to facilitate vector space analysis, such as, forexample, n-gram unit operations. Generally speaking, n-grams reflectcontiguous sequences of items (e.g., features). In some examples, avalue for n is selected to identify a number of occurrences or a valueassociated with a sliding window for string conversion. In someexamples, repeated features of frequency/occurrence n may be ignoredwhen such occurrences are known and/or otherwise believed to beinnocuous, thereby improving log file analysis efficiency.

The example hashing operation builder 316 generates unit operation(s)for hashing of feature types. Returning briefly to the illustratedexample of FIG. 2C, the vector output file 270 represents observedfeatures as different integer values. As described above, the observedfeature “android.os.SystemProperties.get” may be represented as integervalue 13, the observed feature to “CreateFileW” may be represented asinteger value 39, and the observed feature to “CreateFileW” having anassociated executable file name of “popupkiller.exe” may be representedas integer value 115. As such, the example hashing unit operation (e.g.,metadata tag “h01”) generated by the example hashing operation builder316 reflects operations/functions to hash the observed features intoparticular integer values as defined by one or more dictionarydefinitions. As such, in the event a new feature is discovered thatshould be included in machine learning evaluation, a completely newvector creation program to extract the new feature does not have to bedesigned, coded and/or managed by the malware evaluation personnel.Instead, the corresponding dictionary of interest can be updated withthe new feature name/type and a unique integer value for that featuremay be assigned in the dictionary.

To prepare a vector output file that conforms to a format of interest,the example formatting operation builder 318 generates a unit operationfor the target classification format of interest (e.g., metadata tag“f01”). In some examples, the machine learning system 108 must receiveand/or otherwise retrieve input data in a particular format, such asLIBSVM. The example formatting operation builder 318 prepares the outputlog file (e.g., the example vector output file 270 of FIG. 2C) in amanner consistent with the desired classification format.

To save the vector output file upon completion of classificationformatting efforts (e.g., in response to invoking metadata tag ‘f01”),the example feature save operation builder 320 generates an associatedfeature save unit operation that can be invoked by calling, for example,metadata tag “s01.” When saved, the output vector file (e.g., the vectoroutput file 270 of FIG. 2C) may be sent and/or otherwise transmitted tothe example machine learning system 108.

After the example unit operation builder 308 has created and/orotherwise generated one or more unit operations having associatedmetadata tag names, one or more combinations of such unit operations maybe assembled into particular sequences to process a received input logfile of interest (e.g., the example text log 200 of FIG. 2A, the exampletext log 250 of FIG. 2B, an example JSON log, an example binary log,etc.). In operation, the example operation flow builder 322 builds oneor more operation flow sequences for the input type of interest. In someexamples, the example operation flow builder 322 generates a unique typeidentifier and a corresponding name to be referenced for future callswhen processing the input log file of interest. Example operation flowsequences are shown in the illustrated example of Table 2 below.

TABLE 2 Type Other ID Name Op ID Sequence information 1 text p01, m01*,e01*, n01, h01, f01, s01 Pattern string for e01. 2 json p02, m01*, t01*,n01, h01, f01, s01 Dictionary for t01. 3 text2 p01, m01*, e02*, n01,h01, f01, s01

In the illustrated example of Table 2, the operation flow builder 322associates a first Type_ID value of “1” with a sequence name “text.”Additionally, the example operation flow builder 322 associates theType_ID and name with a particular Op_ID sequence of interest, in whichmetadata tags are invoked in a particular order of “p01, m01, e01, n01,h01, f01 and s01.” In the illustrated example of Table 2, an asterisk(“*”) indicates that the functionality associated with the correspondingmetadata tag is to be repeated as needed to accomplish a task (e.g.,repeated in a loop to identify all nomenclature matches of interest froma dictionary). For example, the operation flow associated with Type_ID“1” loops the example unit operation “m01” to find one or more matchesvia regular expression string(s) and/or dictionary matches.

The example operation flow builder 322 establishes metadata sequenceplacement for file to string unit operation (e.g., assigning a metadataunit operation such as “p01”), followed by metadata sequence placementfor extraction, matching and/or translation (e.g., assigning metadataunit operation(s) such as “e01,” “m01,” and/or “t01,” respectively. Theexample operation flow builder 322 establishes metadata sequenceplacement for vector space analysis operation(s), such as metadata unitoperation “n01”, and establishes sequence placement for hashingoperation(s) (e.g., metadata unit operation “h01”). The exampleoperation flow builder 322 also establishes metadata sequence placementfor target classification formatting operation(s) (e.g., metadata unitoperation “f01”), and establishes metadata sequence placement for savingcompleted vector output files to a memory (e.g., metadata unit operation“s01”).

In the event one or more additional and/or alternate input logs areknown and/or otherwise available, the example feature engineering system110 may (a) prepare dictionary storage 304 and regular expressionstorage 306 to accommodate for feature nomenclature to be used and/orotherwise expected in the log, (b) build one or more unit operations tohandle the input log(s) (e.g., as described above in connection withexample Table 1) and (c) build one or more operation flow sequences forinput type(s) of interest (e.g., as described above in connection withexample Table 2). However, during runtime, the example featureengineering system 110 may invoke the example log file retriever 324 toretrieve and/or otherwise receive an input log file (e.g., retrieve thelog file from the example behavior aggregator 102 of FIG. 1). If theexample log file retriever 324 does not recognize the retrieved log filetype, then the example feature engineering system 110 operates in aconfiguration mode to (a) prepare the example dictionary storage 304and/or regular expression storage 306 to accommodate for featurenomenclature to be used and/or otherwise expected in the log, (b) buildone or more unit operations to handle the input log(s), and (c) buildone or more operation flow sequences for input type(s) of interest.

On the other hand, in the event the example log file retriever 324recognizes the log file type (e.g., a log file type of text that has anassociated dictionary, regular expression string library, one or moreunit operations and one or more operation flow sequence(s)), then theexample operation flow builder 322 selects an appropriate operation flowsequence. For example, in the event the input log is associated with afirst text type, then the example operation flow builder 322 identifiesa matching “text” name and the associated Type_ID “1” to extract thecorrect operation flow sequence for the input log of interest. In thisexample, the corresponding operation flow sequence is“p01,m01*,e01,n01,h01,f01 and s01.” On the other hand, in the event theretrieved log file type is associated with “json,” then the exampleoperation flow builder 322 retrieves operation flow sequence associatedwith Type_ID “2.”

The example dictionary editor 302 retrieves and/or otherwise identifiescorresponding dictionaries and/or regular expression string(s) that areassociated with the selected operation flow sequence, and the exampleoperation flow builder 322 executes the corresponding operation flowsequence to process the input log file. When complete, the examplefeature engineering system 110 has a stored output file, such as theexample output vector file 270 of FIG. 2C, which may be sent to a vectorfeature analyzer, such as the example machine learning system 108 ofFIG. 1.

While an example manner of implementing the feature engineering system110 of FIG. 1 is illustrated in FIGS. 1 and 3, one or more of theelements, processes and/or devices illustrated in FIG. 3 may becombined, divided, re-arranged, omitted, eliminated and/or implementedin any other way. Further, the example dictionary editor 302, theexample dictionary storage 304, the example regular expression storage306, the example unit operation builder 308, the example file to stringoperation builder 310, the example extraction operation builder 312, theexample vector space operation builder 314, the example hashingoperation builder 316, the example formatting operation builder 318, theexample feature save operation builder 320, the example operation flowbuilder 322, the example log file retriever 324 and/or, more generally,the example feature engineering system 110 of FIGS. 1 and 3 may beimplemented by hardware, software, firmware and/or any combination ofhardware, software and/or firmware. Thus, for example, any of theexample dictionary editor 302, the example dictionary storage 304, theexample regular expression storage 306, the example unit operationbuilder 308, the example file to string operation builder 310, theexample extraction operation builder 312, the example vector spaceoperation builder 314, the example hashing operation builder 316, theexample formatting operation builder 318, the example feature saveoperation builder 320, the example operation flow builder 322, theexample log file retriever 324 and/or, more generally, the examplefeature engineering system 110 of FIGS. 1 and 3 could be implemented byone or more analog or digital circuit(s), logic circuits, programmableprocessor(s), application specific integrated circuit(s) (ASIC(s)),programmable logic device(s) (PLD(s)) and/or field programmable logicdevice(s) (FPLD(s)). When reading any of the apparatus or system claimsof this patent to cover a purely software and/or firmwareimplementation, at least one of the example dictionary editor 302, theexample dictionary storage 304, the example regular expression storage306, the example unit operation builder 308, the example file to stringoperation builder 310, the example extraction operation builder 312, theexample vector space operation builder 314, the example hashingoperation builder 316, the example formatting operation builder 318, theexample feature save operation builder 320, the example operation flowbuilder 322, the example log file retriever 324 and/or, more generally,the example feature engineering system 110 of FIGS. 1 and 3 is/arehereby expressly defined to include a tangible computer readable storagedevice or storage disk such as a memory, a digital versatile disk (DVD),a compact disk (CD), a Blu-ray disk, etc. storing the software and/orfirmware. Further still, the example feature engineering system 110 ofFIGS. 1 and/or 3 may include one or more elements, processes and/ordevices in addition to, or instead of, those illustrated in FIG. 3,and/or may include more than one of any or all of the illustratedelements, processes and devices.

Flowcharts representative of example machine readable instructions forimplementing the feature engineering system 110 of FIGS. 1 and 3 areshown in FIGS. 4-6. In these examples, the machine readable instructionscomprise a program for execution by a processor such as the processor712 shown in the example processor platform 700 discussed below inconnection with FIG. 7. The program may be embodied in software storedon a tangible computer readable storage medium such as a CD-ROM, afloppy disk, a hard drive, a digital versatile disk (DVD), a Blu-raydisk, or a memory associated with the processor 712, but the entireprogram and/or parts thereof could alternatively be executed by a deviceother than the processor 712 and/or embodied in firmware or dedicatedhardware. Further, although the example program is described withreference to the flowcharts illustrated in FIGS. 4-6, many other methodsof implementing the example feature engineering system 110 mayalternatively be used. For example, the order of execution of the blocksmay be changed, and/or some of the blocks described may be changed,eliminated, or combined.

As mentioned above, the example processes of FIGS. 4-6 may beimplemented using coded instructions (e.g., computer and/or machinereadable instructions) stored on a tangible computer readable storagemedium such as a hard disk drive, a flash memory, a read-only memory(ROM), a compact disk (CD), a digital versatile disk (DVD), a cache, arandom-access memory (RAM) and/or any other storage device or storagedisk in which information is stored for any duration (e.g., for extendedtime periods, permanently, for brief instances, for temporarilybuffering, and/or for caching of the information). As used herein, theterm tangible computer readable storage medium is expressly defined toinclude any type of computer readable storage device and/or storage diskand to exclude propagating signals and to exclude transmission media. Asused herein, “tangible computer readable storage medium” and “tangiblemachine readable storage medium” are used interchangeably. Additionallyor alternatively, the example processes of FIGS. 4-6 may be implementedusing coded instructions (e.g., computer and/or machine readableinstructions) stored on a non-transitory computer and/or machinereadable medium such as a hard disk drive, a flash memory, a read-onlymemory, a compact disk, a digital versatile disk, a cache, arandom-access memory and/or any other storage device or storage disk inwhich information is stored for any duration (e.g., for extended timeperiods, permanently, for brief instances, for temporarily buffering,and/or for caching of the information). As used herein, the termnon-transitory computer readable medium is expressly defined to includeany type of computer readable storage device and/or storage disk and toexclude propagating signals and to exclude transmission media. As usedherein, when the phrase “at least” is used as the transition term in apreamble of a claim, it is open-ended in the same manner as the term“comprising” is open ended.

The program 400 of FIG. 4 begins at block 402, where the example featureengineering system 110 determines whether to operate in a manner thatfacilitates configuration to handle input log file types, or whether tooperate in a runtime manner that processes retrieved and/or otherwisereceived input log file types. In the event the example featureengineering system 110 is to configure itself to be prepared to aparticular input log file type (block 402), the example dictionaryeditor 302 updates the example dictionary storage 304 and/or the exampleregular expression storage 306 with feature nomenclature definitions,feature nomenclature combination(s) and/or corresponding regularexpression string(s) associated with the input log type of interest(block 404). As described above, the example dictionary editor 302 mayparse a newly received input log file for new nomenclature and/ornomenclature combinations and set corresponding dictionary definitionsand/or regular expression strings to extract such indications of featureoccurrences.

For example, the dictionary editor 302 may retrieve the example text log200 of FIG. 2A and set a first feature name as “android.os.SystemProperties.get” to be associated with a previously unused listof integers (block 404). As described above, the example vector outputfile 270 includes a vector list, in which every unique feature has anassociated unique integer representation. The example dictionary editor302 assigns the newly identified feature to a next available integervalue so that any future reference to the feature may be associated withthat uniquely assigned integer value. In other examples, the dictionaryeditor determines which two or more occurrences of nomenclature are tobe associated with a feature instance. For example, a combination of theterm “java” plus “net” plus “Socket” can refer to several differentfeatures that include a concatenated string “java.netSocket.” However,the example dictionary editor 302 only assigns a feature if theaforementioned substring is also appended with another term, such as“getlnputStream” to indicate a feature occurrence associated with inputstream retrieval in a socket communication. In another example, theexample dictionary editor 302 assigns an alternate feature in responseto detecting the appended nomenclature “getOutputStream” to indicate afeature occurrence associated with output stream retrieval in a socketcommunication. As described above and in further detail below, theexample unit operation builder 308 generates a metadata tag andassociated operational functionality to parse an input string to findone or more occurrences of features (block 406), such as occurrences of“java.netSocketgetInputStream” and/or ‘java.net.Socket.getOutputStream.”

The example unit operation builder 308 builds unit operations to beassociated with the input log of interest (block 406). FIG. 5illustrates additional detail associated with building unit operations(block 406) of FIG. 4. While the illustrated example of FIG. 5 considersa “top down” approach to building unit operations, examples are notlimited thereto. In some examples, respective ones of the operation(s)of FIG. 5 may be invoked, as needed. For instance, one or more portionsof FIG. 5 may be invoked to accomplish respective operations during oneor more subsequent iteration(s). In the illustrated example of FIG. 5,the example file to string operation builder 310 generates a file tostring unit operation (block 502) to convert the log file of interest toa string. For example, the file to string operation builder 310 mayassign a tag named “p01” with operations (e.g., string manipulationcode) to convert each row of the log file to a string, which can laterbe evaluated by one or more other operations for extraction, matching,hashing, etc., as described above and in further detail below. When theexample log file of interest has been converted to a string (block 502),the example extraction operation builder 312 generates one or more unitoperations and associated metadata tags to extract one or moreexpressions of interest (block 504). As described above, the extractionoperation builder 312 may generate a metadata tag named “m01” to find amatching feature using regular expression string(s) stored in theexample regular expression storage 306, and/or identify matchingfeature(s) by referencing the example dictionary storage 304. Withoutlimitation, the example extraction operation builder 312 may generate ametadata tag named “e01” to extract one or more particular substrings.In some examples, slightly different nomenclature in thecandidate/target input log file refers to similar features, for whichthe example extraction operation builder 312 may normalize via ametadata tag named “t01” associated with translation logic, as describedabove.

The example vector space operation builder 314 generates a unitoperation for vector space analysis (block 506), such as operations thatemploy n-gram logic associated with a metadata tag named “n01.” Theexample hashing operation builder 316 generates a unit operation tofacilitate hashing (block 508), such as operations to hash and/orotherwise associate one or more features into an integer representation.As described above, the example vector output file 270 represents eachfeature as a unique integer value, associations for which may be storedin the example dictionary storage 304 to be assigned during hashingoperation(s) (block 508). The example formatting operation builder 318generates a unit operation for formatting a vector output file into aclassification intended for a target analysis effort (block 510), suchas efforts performed by the example machine learning system 108 ofFIG. 1. One example format is the LIBSVM format, but examples disclosedherein are not limited thereto. The example formatting operation builder318 may assign a metadata tag for this operation, such as “f01.” Tofacilitate saving the vector output file, the example feature saveoperation builder 320 generates a unit operation for saving an outputfile (block 512). In some examples, the feature save operation builder320 assigns a metadata tag name “s01” that may be called as part of ametadata tag sequence, as described above and below. Control thenreturns to block 408 of FIG. 4.

Returning to the illustrated example of FIG. 4, the example featureengineering system 110 now has unit operations that are tailored and/orotherwise unique to a particular input log of interest. Such unitoperations may be called by referencing their associated metadata tagnomenclature into a particular sequence to build a vector output file(e.g., the vector output file 270 of FIG. 2C) that can be provided toone or more machine learning systems (e.g., the example machine learningsystem 108 of FIG. 1). The example operation flow builder 322 builds oneor more operation flow sequences for the input log of interest (block408).

In the illustrated example of FIG. 6, the operation flow builder 322generates a unique type identifier and associated name for the input logof interest (block 602). As described above in connection with Table 2,each input log of interest may have one or more operation flow sequencesfor which each candidate sequence is assigned an associated name (e.g.,“text” to refer to an input log of type text data, “json” to refer to aninput log of type JSON data, etc.) and an associated Type_ID. Theexample operation flow builder 322 initially establishes a sequenceplacement metadata tag for a file to string unit operation (block 604),such as “p01” to invoke operations related to converting the input logof type text into a string for later processing. Such metadata tags maybe sequenced, assembled and/or otherwise built via a user interface or atext editor, in which the malware evaluation personnel can edit. Withthe data from the input log in a string format, the example operationflow builder 322 establishes sequence to facilitate one or more offeature extraction, feature matching and/or feature translation (block606). For example, the input log may utilize one or more dictionariesthat are accessed by operational logic in response to calling themetadata tag “m01.” The called metadata tag may operate in a loop asmany times as needed to search through the string for matching ones offeatures identified in the example input log (sometimes signified by anasterisk (“*”). In some examples, the operation flow builder 322facilitates one or more sequence placements for vector space analysis(block 608), such as operations to perform n-grams.

The example operation flow builder 322 establishes a sequence tofacilitate hashing operation(s) (block 610), which allow the relativelylong feature description nomenclature to be represented by uniqueinteger values. Because each target machine learning system may have aparticular classification input format needed for machine learningactivity, the example operation flow builder 322 establishes a sequenceto facilitate target classification formatting (block 612). As describedabove, one example classification format includes LIBSVM, but examplesdisclosed herein are not limited thereto. To allow saving a vectoroutput file, the example operation flow builder 322 establishes sequenceplacement for a feature save operation (block 614) (e.g., “s01”).

In some examples, the malware evaluation personnel may attempt to buildone or more vector output files that utilize alternate sequences ofmetadata-driven operations. For example, some input log files mayrequire different combinations of extraction, matching and/ortranslation that utilize one or more dictionaries and/or regularexpression string(s). In the event one or more additional flowsequence(s) are to be created (block 616), control returns to block 602.The example operation flow builder 322 also verifies that an assembledsequence is complete (block 618). For example, in the event the malwareevaluation personnel created the assembled sequence by editing a textfile of one or more metadata tags, the example operation flow builder322 verifies that one or more metadata tags is not missing (e.g., themalware evaluation personnel did not select a metadata tag to performfeature matching). If one or more metadata tags is deemed missing (block618), the example operation flow builder 322 generates a prompt that thesequence includes one or more errors or missing metadata tags (block620), and control is advanced to a respective block to identify whichmetadata tag should be added and/or otherwise checked (e.g., one or moreof blocks 604-614).

Returning to the illustrated example of FIG. 4, the example featureengineering system 110 may also operate in a runtime mode (block 402).If so, the example log file retriever 324 retrieves and/or otherwisereceives an input log file (block 410) and determines if the receivedinput log file is recognized (block 412). For instance, a new input logfile may have been received that had not previously been evaluatedduring a configuration mode of the example feature engineering system110. If so, control advances to block 404 to configure the featureengineering system 110 in view of the previously unseen input log file.

If the received and/or otherwise retrieved input log file is recognized(block 412), the example operation flow builder 322 selects an operationflow sequence that is associated with the input log file of interest(block 414). In some examples, the operation flow builder 322 analyzesthe input log file to determine a type of “text,” “json,” etc.Additionally, the example operation flow builder 322 selects a candidateoperation flow sequence for that particular input log file type so thatmetadata tags associated therewith can be executed in their particularsequential order. The example dictionary editor 302 retrieves and/orotherwise identifies corresponding dictionaries (e.g., one or moredictionaries stored in the example dictionary storage 304) and/orcorresponding regular expression string(s) (e.g., one or more regularexpression string(s) stored in the example regular expression storage306) (block 416). The example operation flow builder 322 executes theselected operation flow sequence (block 418) based on the combination ofmetadata tags associated with that selected flow sequence to generate avector output file, such as the example vector output file 270illustrated in FIG. 2C. Once the vector output file has been created,the example feature engineering system 110 sends the vector output fileto one or more feature analyzer(s) (block 420), such as the examplemachine learning system 108 of FIG. 1.

FIG. 7 is a block diagram of an example processor platform 700 capableof executing the instructions of FIGS. 4-6 to implement the featureengineering system 110 of FIGS. 1 and 3. The processor platform 700 canbe, for example, a server, a personal computer, an Internet appliance,or any other type of computing device.

The processor platform 700 of the illustrated example includes aprocessor 712. The processor 712 of the illustrated example is hardware.For example, the processor 712 can be implemented by one or moreintegrated circuits, logic circuits, microprocessors or controllers fromany desired family or manufacturer. In the illustrated example of FIG.7, the processor 700 includes one or more example processing cores 715configured via example instructions 732, which include the exampleinstructions of FIGS. 4-6 to implement the example feature engineeringsystem 110 of FIGS. 1 and/or 3.

The processor 712 of the illustrated example includes a local memory 713(e.g., a cache). The processor 712 of the illustrated example is incommunication with a main memory including a volatile memory 714 and anon-volatile memory 716 via a bus 718. The volatile memory 714 may beimplemented by Synchronous Dynamic Random Access Memory (SDRAM), DynamicRandom Access Memory (DRAM), RAMBUS Dynamic Random Access Memory (RDRAM)and/or any other type of random access memory device. The non-volatilememory 716 may be implemented by flash memory and/or any other desiredtype of memory device. Access to the main memory 714, 716 is controlledby a memory controller.

The processor platform 700 of the illustrated example also includes aninterface circuit 720. The interface circuit 720 may be implemented byany type of interface standard, such as an Ethernet interface, auniversal serial bus (USB), and/or a PCI express interface.

In the illustrated example, one or more input devices 722 are connectedto the interface circuit 720. The input device(s) 722 permit(s) a userto enter data and commands into the processor 712. The input device(s)can be implemented by, for example, an audio sensor, a microphone, acamera (still or video), a keyboard, a button, a mouse, a touchscreen, atrack-pad, a trackball, isopoint and/or a voice recognition system.

One or more output devices 724 are also connected to the interfacecircuit 720 of the illustrated example. The output devices 724 can beimplemented, for example, by display devices (e.g., a light emittingdiode (LED), an organic light emitting diode (OLED), a liquid crystaldisplay, a cathode ray tube display (CRT), a touchscreen, a tactileoutput device, a printer and/or speakers). The interface circuit 720 ofthe illustrated example, thus, typically includes a graphics drivercard, a graphics driver chip or a graphics driver processor.

The interface circuit 720 of the illustrated example also includes acommunication device such as a transmitter, a receiver, a transceiver, amodem and/or network interface card to facilitate exchange of data withexternal machines (e.g., computing devices of any kind) via a network726 (e.g., an Ethernet connection, a digital subscriber line (DSL), atelephone line, coaxial cable, a cellular telephone system, etc.).

The processor platform 700 of the illustrated example also includes oneor more mass storage devices 728 for storing software and/or data.Examples of such mass storage devices 728 include floppy disk drives,hard drive disks, compact disk drives, Blu-ray disk drives, RAIDsystems, and digital versatile disk (DVD) drives. In some examples, themass storage device 728 may implement the example dictionary storage 304and/or the example regular expression storage 306.

The coded instructions 732 of FIGS. 4-6 may be stored in the massstorage device 728, in the volatile memory 714, in the non-volatilememory 716, and/or on a removable tangible computer readable storagemedium such as a CD or DVD.

Example methods, apparatus, systems and articles of manufacture toimprove feature engineering efficiency with metadata unit operations aredisclosed herein. Further examples and combinations thereof include thefollowing.

Example 1 is a computer-implemented method to apply feature engineeringwith metadata-driven unit operations, including retrieving a log file ina first file format, the log file containing feature occurrence data,generating a first unit operation based on the first file format toextract the feature occurrence data from the log file to a string, thefirst unit operation associated with a first metadata tag, generatingsecond unit operations to identify respective features from the featureoccurrence data, the second unit operations associated with respectivesecond metadata tags, and generating a first sequence of the firstmetadata tag and the second metadata tags to create a first vectoroutput file of the feature occurrence data.

Example 2 includes the method as defined in example 1, wherein the firstunit operation includes parsing operations for at least one of a textfile format, a comma separated value (CSV) file format, a JavaScriptObject Notation (JSON) file format, or a binary file format.

Example 3 includes the method as defined in example 1, further includingbuilding a dictionary of feature nomenclature associated with therespective features from the feature occurrence data.

Example 4 includes the method as defined in example 3, further includinggenerating search substrings of the feature nomenclature.

Example 5 includes the method as defined in example 4, whereinrespective ones of the second unit operations identify the searchsubstrings as feature occurrence instances.

Example 6 includes the method as defined in example 1, further includingexecuting the first sequence of the first metadata tag and the secondmetadata tags to create the first vector output file associated with thefirst file format, and executing a second sequence of the first metadatatag and alternate ones of the second metadata tags to create a secondvector output file associated with the first file format.

Example 7 includes the method as defined in example 6, wherein thesecond metadata tags invoke a dictionary to identify feature occurrenceinstances, and the alternate ones of the second metadata tags invokeregular expressions to identify feature occurrence instances.

Example 8 includes the method as defined in claim 1, wherein respectiveones of the second unit operations extract feature instances based on atleast one of dictionary matching or regular expression strings.

Example 9 includes the method as defined in example 1, whereinrespective ones of the second unit operations normalize featurenomenclature based on a dictionary association.

Example 10 includes the method as defined in example 1, whereinrespective ones of the second unit operations hash identified featuresto a unique integer value.

Example 11 includes the method as defined in example 1, whereinrespective ones of the second unit operations format the vector outputfile based on a Library for Support Vector Machines (LIBSVM)classification format.

Example 12 is an apparatus to apply feature engineering withmetadata-driven unit operations, comprising a log file retriever toretrieve a log file in a first file format, the log file containingfeature occurrence data, a file to string operation builder to generatea first unit operation based on the first file format to extract thefeature occurrence data from the log file to a string, the first unitoperation associated with a first metadata tag, an extraction operationbuilder to generate second unit operations to identify respectivefeatures from the feature occurrence data, the second unit operationsassociated with respective second metadata tags, and an operation flowbuilder to generate a first sequence of the first metadata tag and thesecond metadata tags to create a first vector output file of the featureoccurrence data.

Example 13 includes the apparatus as defined in example 12, wherein thefile to string operation builder is to generate parsing operations forat least one of a text file format, a comma separated value (CSV) fileformat, a JavaScript Object Notation (JSON) file format, or a binaryfile format.

Example 14 includes the apparatus as defined in example 12, furtherincluding a dictionary editor to build a dictionary of featurenomenclature associated with the respective features from the featureoccurrence data.

Example 15 includes the apparatus as defined in example 14, wherein theextraction operation builder is to generate search substrings of thefeature nomenclature.

Example 16 includes the apparatus as defined in example 15, wherein thesearch substrings identify respective ones of the second unit operationsas feature occurrence instances.

Example 17 includes the apparatus as defined in example 12, wherein theoperation flow builder is to execute the first sequence of the firstmetadata tag and the second metadata tags to create the first vectoroutput file associated with the first file format, and execute a secondsequence of the first metadata tag and alternate ones of the secondmetadata tags to create a second vector output file associated with thefirst file format.

Example 18 includes the apparatus as defined in example 17, wherein thesecond metadata tags invoke a dictionary to identify feature occurrenceinstances, and the alternate ones of the second metadata tags invokeregular expressions to identify feature occurrence instances.

Example 19 includes the apparatus as defined in example 12, furtherincluding a dictionary editor to facilitate extraction of featureinstances from respective ones of the second unit operations based on atleast one of dictionary matching or regular expression strings.

Example 20 includes the apparatus as defined in example 12, furtherincluding a dictionary editor to normalize respective ones of the secondunit operations to identify feature nomenclature based on a dictionaryassociation.

Example 21 includes the apparatus as defined in example 12, furtherincluding a hashing operation builder to hash respective ones of thesecond unit operations to a unique integer value.

Example 22 includes the apparatus as defined in example 12, furtherincluding a formatting operation builder to format the vector outputfile based on a Library for Support Vector Machines (LIBSVM)classification format.

Example 23 is a tangible computer readable storage medium comprisingcomputer readable instructions which, when executed, cause a processorto at least retrieve a log file in a first file format, the log filecontaining feature occurrence data, generate a first unit operationbased on the first file format to extract the feature occurrence datafrom the log file to a string, the first unit operation associated witha first metadata tag, generate second unit operations to identifyrespective features from the feature occurrence data, the second unitoperations associated with respective second metadata tags, and generatea first sequence of the first metadata tag and the second metadata tagsto create a first vector output file of the feature occurrence data.

Example 24 includes the computer readable storage medium of example 23,wherein the instructions, when executed, cause the processor to generateparsing operations for at least one of a text file format, a commaseparated value (CSV) file format, a JavaScript Object Notation (JSON)file format, or a binary file format.

Example 25 includes the computer readable storage medium of example 23,wherein the instructions, when executed, cause the processor to build adictionary of feature nomenclature associated with the respectivefeatures from the feature occurrence data.

Example 26 includes the computer readable storage medium of example 25,wherein the instructions, when executed, cause the processor to generatesearch substrings of the feature nomenclature.

Example 27 includes the computer readable storage medium of example 26,wherein the instructions, when executed, cause the processor toidentify, from respective ones of the second unit operations, the searchsubstrings as feature occurrence instances.

Example 28 includes the computer readable storage medium of example 23,wherein the instructions, when executed, cause the processor to executethe first sequence of the first metadata tag and the second metadatatags to create the first vector output file associated with the firstfile format, and execute a second sequence of the first metadata tag andalternate ones of the second metadata tags to create a second vectoroutput file associated with the first file format.

Example 29 includes the computer readable storage medium of example 28,wherein the instructions, when executed, cause the processor to invoke,via the second metadata tags, a dictionary to identify featureoccurrence instances, and the alternate ones of the second metadata tagsinvoke regular expressions to identify feature occurrence instances.

Example 30 includes the computer readable storage medium of example 23,wherein the instructions, when executed, cause the processor to extract,from respective ones of the second unit operations, feature instancesbased on at least one of dictionary matching or regular expressionstrings.

Example 31 includes the computer readable storage medium of example 23,wherein the instructions, when executed, cause the processor tonormalize, from respective ones of the second unit operations, featurenomenclature based on a dictionary association.

Example 32 includes the computer readable storage medium of claim 23,wherein the instructions, when executed, cause the processor to hash,from respective ones of the second unit operations, identified featuresto a unique integer value.

Example 33 includes the computer readable storage medium of example 23,wherein the instructions, when executed, cause the processor to format,from respective ones of the second unit operations, the vector outputfile based on a Library for Support Vector Machines (LIBSVM)classification format.

Example 34 is a system to apply feature engineering with metadata-drivenunit operations, comprising means for retrieving a log file in a firstfile format, the log file containing feature occurrence data, means forgenerating a first unit operation based on the first file format toextract the feature occurrence data from the log file to a string, thefirst unit operation associated with a first metadata tag, means forgenerating second unit operations to identify respective features fromthe feature occurrence data, the second unit operations associated withrespective second metadata tags, and means for generating a firstsequence of the first metadata tag and the second metadata tags tocreate a first vector output file of the feature occurrence data.

Example 35 includes the system as defined in example 34, furtherincluding means for generating parsing operations for at least one of atext file format, a comma separated value (CSV) file format, aJavaScript Object Notation (JSON) file format, or a binary file format.

Example 36 includes the system as defined in example 34, furtherincluding means for building a dictionary of feature nomenclatureassociated with the respective features from the feature occurrencedata.

Example 37 includes the system as defined in example 36, furtherincluding means for generating search substrings of the featurenomenclature.

Example 38 includes the system as defined in example 37, furtherincluding means for identifying respective ones of the second unitoperations as feature occurrence instances.

Example 39 includes the system as defined in example 34, furtherincluding means for executing the first sequence of the first metadatatag and the second metadata tags to create the first vector output fileassociated with the first file format, and executing a second sequenceof the first metadata tag and alternate ones of the second metadata tagsto create a second vector output file associated with the first fileformat.

Example 40 includes the system as defined in example 39, furtherincluding means for invoking a dictionary to identify feature occurrenceinstances, and the alternate ones of the second metadata tags invokeregular expressions to identify feature occurrence instances.

Example 41 includes the system as defined in example 34, furtherincluding means for facilitating extraction of feature instances fromrespective ones of the second unit operations based on at least one ofdictionary matching or regular expression strings.

Example 42 includes the system as defined in example 34, furtherincluding means for normalizing respective ones of the second unitoperations to identify feature nomenclature based on a dictionaryassociation.

Example 43 includes the system as defined in example 34, furtherincluding means for hashing respective ones of the second unitoperations to a unique integer value.

Example 44 includes the system as defined in example 34, furtherincluding a formatting operation builder to format the vector outputfile based on a Library for Support Vector Machines (LIBSVM)classification format.

From the foregoing, it will be appreciated that the above disclosedmethods, apparatus and articles of manufacture reduce a need to developand maintain disparate programs for each type of file format that may begenerated by behavior aggregators that are chartered with theresponsibility of collecting feature behavior associated with programsexecuting on computing devices. In particular, traditional techniques togenerate vector output files suitable for machine learning systemsrequired the development of unique file parsing programs depending oneach file format type, such as output files from the behavioraggregators as text files, JSON files, CSV files and/or binary files. Inthe event a new type of feature having a new nomenclature is identified,then malware evaluation personnel needed to identify each correspondingprogram and modify it to accommodate for the new nomenclature and/or newcombinations of existing nomenclature. Such management and maintenanceefforts are error prone and require duplicative efforts on all fileextraction programs under their control. Examples disclosed hereinreduce such duplicative efforts and reduce potential maintenance errorsby facilitating a metadata-driven approach to log file processing inwhich a dictionary can be used as a repository for feature nomenclature,feature nomenclature combinations, and metadata tags to invoke log fileprocessing operations in a centralized manner.

Although certain example methods, apparatus and articles of manufacturehave been disclosed herein, the scope of coverage of this patent is notlimited thereto. On the contrary, this patent covers all methods,apparatus and articles of manufacture fairly falling within the scope ofthe claims of this patent.

1.-34. (canceled)
 35. A non-transitory computer readable storage mediumcomprising instructions which, when executed, improve an efficiency of amachine learning algorithm by causing one or more processors to atleast: retrieve a log file in a first format, the log file containingbehavior-related data to be analyzed by one or more machine learningalgorithms; convert the log file from the first format to a secondformat; prior to machine learning algorithm application, improve machinelearning modeling efficiency by distinguishing candidate maliciousfeatures from non-malicious features by extracting respectivebehavior-related features from the behavior-related data of the log filein the second format based on the respective ones of thebehavior-related features matching one or more patterns corresponding tomalware; hash the extracted behavior-related features corresponding tomalware; format input data for the machine learning algorithm bycreating a vector output file of the hashed respective ones of thebehavior-related features extracted from the behavior-related datacorresponding to malware; and improve the efficiency of the machinelearning algorithm by transmitting the formatted vector output file to asystem executing the machine learning algorithm.
 36. The non-transitorycomputer readable storage medium of claim 35, wherein thebehavior-related data is generated by a computing device executingsoftware related to malware prevention.
 37. The non-transitory computerreadable storage medium of claim 35, wherein the behavior-related dataincludes a representation of a sequence of system events occurringduring execution of a program on a computing device.
 38. Thenon-transitory computer readable storage medium of claim 35, wherein thefirst format corresponds to at least one of a text file format, a commaseparated value file format, a JavaScript Object Notation file format, abinary file format, or an Extensible Markup Language file format. 39.The non-transitory computer readable storage medium of claim 35, whereinthe second format corresponds to a string format.
 40. The non-transitorycomputer readable storage medium of claim 35, wherein the instructions,when executed, cause the one or more processors to execute a unitoperation based on the first format to convert the log file from thefirst format to the second format.
 41. The non-transitory computerreadable storage medium of claim 40 wherein the unit operation includesstring manipulation code.
 42. The non-transitory computer readablestorage medium of claim 35, wherein the one or more patternscorresponding to malware include a regular expression that causesextraction of the behavior related features from the log fileindependent of the first format of the log file.
 43. The non-transitorycomputer readable storage medium of claim 42, wherein the regularexpression includes metacharacters indicative of at least one of Booleanoperations, grouping operations, or quantification operations.
 44. Anapparatus to improve an efficiency of a machine learning algorithm, theapparatus comprising: a log file retriever to retrieve a log file in afirst format, the log file containing behavior-related data to beanalyzed by one or more machine learning algorithms; an operation flowbuilder to: convert the log file from the first format to a secondformat; prior to machine learning algorithm application, improve machinelearning modeling efficiency by distinguishing candidate maliciousfeatures from non-malicious features by extracting respectivebehavior-related features from the behavior-related data of the log filein the second format based on the respective ones of thebehavior-related features matching one or more patterns corresponding tomalware; hash the extracted behavior-related features corresponding tomalware; and format input data for the machine learning algorithm bycreating a vector output file of the hashed respective ones of thebehavior-related features extracted from the behavior-related datacorresponding to malware; and a feature engineering system to improvethe efficiency of the machine learning algorithm by transmitting theformatted vector output file to a system executing the machine learningalgorithm.
 45. The apparatus of claim 44, wherein the behavior-relateddata is generated by a computing device executing software related tomalware prevention.
 46. The apparatus of claim 44, wherein thebehavior-related data includes a representation of a sequence of systemevents occurring during execution of a program on a computing device.47. The apparatus of claim 44, wherein the log file retriever is toidentify the first format as at least one of a text file format, a commaseparated value file format, a JavaScript Object Notation file format, abinary file format, or an Extensible Markup Language file format. 48.The apparatus of claim 44, wherein the second format corresponds to astring format.
 49. The apparatus of claim 44, wherein the operation flowbuilder is to execute a unit operation based on the first format toconvert the log file from the first format to the second format.
 50. Theapparatus of claim 49, wherein the unit operation includes stringmanipulation code.
 51. The apparatus of claim 44, wherein the one ormore patterns corresponding to malware include a regular expression thatcauses extraction of the behavior related features from the log fileindependent of the first format of the log file.
 52. The apparatus ofclaim 51, wherein the regular expression includes metacharactersindicative of at least one of Boolean operations, grouping operations,or quantification operations.
 53. A method to improve an efficiency of amachine learning algorithm, the method comprising: retrieving a log filein a first format, the log file containing behavior-related data to beanalyzed by one or more machine learning algorithms; converting the logfile from the first format to a second format; prior to machine learningalgorithm application, improving machine learning modeling efficiency bydistinguishing candidate malicious features from non-malicious featuresby extracting respective behavior-related features from thebehavior-related data of the log file in the second format based on therespective ones of the behavior-related features matching one or morepatterns corresponding to malware; hashing the extractedbehavior-related features corresponding to malware; formatting inputdata for the machine learning algorithm by creating a vector output fileof the hashed respective ones of the behavior-related features extractedfrom the behavior-related data corresponding to malware; and improvingthe efficiency of the machine learning algorithm by transmitting theformatted vector output file to a system executing the machine learningalgorithm.
 54. The method of claim 53, wherein the behavior-related datais generated by a computing device executing software related to malwareprevention.